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Abstract 

Learning-by-teaching has been identified as one of the more effective approaches to learning. 
Recently, educational researchers have investigated virtual environments in order to utilize the 
learning-by-teaching pedagogy. In a face-to-face learning-by-teaching situation, the role of the 
learners is to teach their peers or instructors. In virtual environments, learners take an active 
role by teaching a computer agent, which is referred to as Teachable Agent (TA). Although the 
current TA systems have shown their effectiveness on students' learning, there are some 
challenges associated with learner-computer interaction methods. One of the most popular 
interaction methods between the learner and the system is a concept map approach. The 
learner teaches TA by creating information structures by drawing and editing their concept map. 
However, the learner can teach TA rather constrained topics, such as concept-related materials 
or causal effects. It is difficult for TA systems to be utilized in different types of learning along 
with concept-related areas. Therefore, new approaches or methods for communication 
between a human learner and TA systems are required. This project aims to suggest a virtual 
learning-by-teaching environment. A communication method (i.e., a symbol manipulation 
approach) was adopted in this system. The method facilitates the interaction between the 
learner and the computer agent, specifically for K-12 students' mathematics learning. The 
design and development process is described, and future research areas are discussed. 
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Introduction 

Teaching others has been identified as an effective learning strategy (Frager & Stern, 1970; Leikin, 
2006; Muis, Psaradellis, Chevrier, Di Leo, & Lajoie, 2016; Tang, Hernandez, & Adams, 2004). 
Researchers and educators have created virtual learning-by-teaching environments using 
pedagogical computer agents, called a Teachable Agent (TA). A pedagogical agent is a type of 
educational software that has human characteristics and/or appearances. Pedagogical agent 
systems are designed to support learners in online learning environments. TA is a type of 
pedagogical agent, and it takes a unique approach, which is different from the methods that are 
employed in traditional agent systems. Traditional agent systems are designed to be demonstrative 
and unidirectional. On the other hand, TA systems have their own knowledge to demonstrate and 
support interactive activities. In a variety of research, TA systems showed their effectiveness on 
students' learning. However, there are some challenges associated with human-computer 
interaction methods which are used in the previous systems. One of the most popular learner- 
system interaction methods is a concept map approach. The learner teaches a TA through creating 
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information structures by drawing and editing their concept map. However, there are some 
limitations in the current method with one such example being human learners are only able to 
teach limited topic areas. This project aims to suggest a virtual learning-by-teaching environment 
that adopts a new communication method. In addition, this project aims to suggest a preliminary 
idea for extending TA systems into a more theorized virtual learning-by-teaching pedagogy. 


Background 

Traditionally, a pedagogical agent is usually embedded in Intelligent Tutoring Systems (ITS), which 
focuses on modeling learners and tailoring feedback to address learners' immediate needs 
(Leelawong et al., 2003). ITS have successfully been adopted in different types of education fields 
(Kulik & Fletcher, 2015). However, there were some conceptual limitations in the earlier age of ITS. 
For example, traditional ITS provide limited feedback to learners, and limited opportunities to 
practice higher-order cognitive skills in complex domains (Biswas et al., 2005). Since ITS are a type 
of system-oriented learning environments, the learners are considered as knowledge recipients in 
a passive way. 

As a new form of ITS, Chan and Baskin (1988) proposed an approach in which a learner can teach 
learning companion systems. The computer companion agent acts as a human companion to the 
learner. The learning companion agent collaborates with or competes against the human learner 
(Chou, Chan, & Lin, 2003). Learning companion systems are more interactive and reciprocal 
software compared to the earlier ITS products. Still, the learners are passive knowledge recipients. 
The learners are not led to take an active role in their learning within learning companion systems. 


Learning-by-Teaching and Teachable Agent System 

Teachable Agent (TA) systems have been suggested as an advanced form of pedagogical computer 
agents. TA systems are built on the learning-by-teaching pedagogy (Okita & Schwartz, 2006), which 
is identified as an effective instructional/learning strategy. In a face-to-face learning-by-teaching 
situation, the role of the learners is to teach others (e.g., their peers or instructors). In TA systems, 
a human learner takes an active role by teaching a computer agent. Using artificial intelligence 
techniques adopted in the system, the agent shows its performance on summative tests according 
to what was taught by the human learner (i.e., tutor's role). 

In sum, TA systems provide the learner engagement using an instructor-learner metaphor (Chase, 
Chin, Oppezzo, & Schwartz, 2009). The learner teaches a TA and then accesses its knowledge by 
asking the agent a question or requesting the agent to solve a problem. The TA builds its own 
knowledge depending on what was taught by the human tutor (i.e., actual learner). The TA answers 
the questions addressed by the human tutor (i.e., actual learner) or human instructor (i.e., coach's 
role) using artificial intelligence techniques (Okita & Schwartz, 2006). Thus, TA systems rely on the 
instruction of their tutors (i.e., human learners), autonomously learn through the algorithm, and 
perform in given tests. Depending on the TA's answers, the human learners can revise their agent's 
knowledge and their own simultaneously (Blair, Schwartz, Biswas, & Leelawong, 2006). 
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One of the most popular TA systems is Betty's Brain. The teachable agent group at Vanderbilt and 
AAA lab at Stanford collaboratively designed and developed the TA system. The agent has been used 
in scientific domains, where the reasoning with cause-effect structures helps students' learning in 
K-12 environments. The system provides three different modes: (1) Teach Mode: a human learner 
teaches the agent by drawing a concept map and accesses instructional materials as needed to 
acquire information for his/her own teaching; (2) Query Mode: the learner asks the agent questions 
using templates, then the agent answers the questions by reasoning with information which the 
learner taught; and (3) Quiz Mode: the learner evaluates how well he/she taught the agent by 
observing the agent's performance on the quiz, and the system makes suggestions that help the TA 
and the learner correct his/her answers (Davis et al., 2003). 

Betty's Brain makes its qualitative reasoning visible through a concept map approach. The approach 
is a graphical representation of a person's topical understanding (Chin, Dohmen, Cheng, Oppezzo, 
Chase, & Schwartz, 2010). The concept map method is one of the widely accepted techniques for 
constructing learners' knowledge (Leelawong et al., 2003). A learner teaches the TA by creating 
information structures by drawing and editing his/her concept map. The agent uses the concept 
map to answer queries generated by the human tutor (i.e., actual learner). That is, once taught, the 
TA reasons with its own knowledge acquired from the user and answers the questions through a 
qualitative reasoning engine that employs path traversal algorithms (Biswas et al., 2005). The 
learner teaches the TA and receives interactive feedback about how well the learner taught the 
agent. 


Effectiveness of Teachable Agent System 

A variety of empirical research has shown the effectiveness of Betty's Brain. Previous studies can be 
categorized into four themes. First, TA researchers focused on the operation in different modes of 
the system (i.e., teach, query, or quiz mode). For example, the results from Leelawong's et al (2003) 
study indicated that both the query and quiz features had beneficial effects on students' learning. 
Second, the agents' responsive behaviors and learners' self-regulation were broadly investigated 
(see Biswas, Leelawong, Belynne, & Adebiyi, 2005; Roscoe, Wagster, & Biswas, 2008). This strand of 
research shows the significant positive effects of self-regulated learning strategies on learners' 
understanding and knowledge transfer in learning. Third, another research area dealt with 
Preparation for Future Learning (PFL; see Bransford & Schwartz, 1999; Chin et al., 2010). PFL refers 
to how well prepared learners are to learn in a new environment. Experimentation studies in K-12 
classrooms demonstrated the success of the TA system, and showed the learners' high PFL level. 
These studies provide evidence that benefits for learning can persist when the children are learning 
new content without the support of technology (Chin et al., 2010). Last, the other research focused 
on the existence of the protege effect. In the study of Chase et al. (2009), the learners were more 
willing to make the effort towards learning on behalf of a computer protege (i.e., the TA condition) 
rather than for themselves (i.e., the avatar condition). The learners put forth greater effort to learn 
for their TAs than for themselves. Through the four types of studies, the virtual learning-by-teaching 
environment has shown its effectiveness. Along with Betty's Brain, other TA systems, such as KORI 
(Kim et al., 2005) and SimStudent (Matsuda et al., 2010) also showed their usefulness and 
effectiveness as an instructional/learning tool. 
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Challenges and Needs 

Although the current TA systems showed their effectiveness on students' learning, there are some 
challenges associated with human-computer interaction methods. Admittedly, the concept map 
method is an effective, efficient, and useful approach for learners to communicate with computer 
agents. The systems that adopt the concept map approach can easily translate the concept map 
that the learner created. For example, the TA system can understand a relationship schema in the 
concept map (e.g., 'is a', 'has a', or 'causes a' relationships). However, the concept map-based TA 
systems are somewhat limited in becoming an expandable learning environment. This is because 
human learners are only able to teach rather constrained topics such as concept-related materials 
or causal effects (e.g., concept learning areas in the science curriculum, such as ecosystems, climate 
change, or thermoregulation). The issue of limited domain in learning is a significant challenge. This 
is because that it is difficult for the system to be utilized in different types of learning areas along 
with concept-related learning. Given the effectiveness of learning-by-teaching in different types of 
learning domains, TA systems must extend their human-computer interaction methods and 
reasoning mechanisms to accommodate students' teaching and learning processes in diverse ways. 
Therefore, new approaches or methods for communication between a human learner and the TA 
system are required. 


This Project 

This project aims to suggest a virtual learning-by-teaching that adopts a new communication 
method. The TA system was designed and developed. As a human-computer interaction method, a 
symbol manipulation approach was adopted in this system. The method facilitates the 
communication between the human learner and computer agent, specifically for K-12 students' 
mathematics learning. The design conditions were explored and identified prior to the development 
stage. The purpose of designing the system was to: (1) demonstrate the TA is working properly; (2) 
find out possible modifications or design conditions for better implementation; and (3) make 
decisions whether it is worth to develop for other subject areas. This project seeks ways to address 
the challenges of the traditional human-computer interaction approaches as reviewed above. 


Design and Development 


Design Conditions 

Initially, three types of design conditions were formulated from the literature review. First, 
Teachable Agent (TA) systems need to provide the learners with more interactive experiences. From 
the learning-by-teaching pedagogical perspective, the learner who takes a tutor's role gains deeper 
understanding from different types of interactions with the tutee. For example, the learner answers 
questions, explains materials, and discovers misconceptions. TA systems need be designed to 
benefit its users in the same fashion (Davis et al., 2003). In order to facilitate the different types of 
interactions, the system must facilitate the human learner in monitoring how the agent answers the 
questions and solves the problems. In addition, users (i.e., learners) should be able to correct what 
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they taught when the users notice any discrepancies between their own knowledge and the agent's. 
Specifically, Blair et al. (2006) provided four guidelines for learning-by-teaching interactions: (1) "use 
explicit and well-structured visual representations"; (2) "enable the agent to take independent 
actions"; (3) "model productive learner behaviors"; and (4) "include environments that support 
teaching interactions" (pp. 3-4). To sum up, TA systems are required to clearly demonstrate qualities 
and characteristics of human learners for supporting the aforementioned interactive activities. 

Second, TA systems must provide more control opportunities for learners. As Kim et al. (2005) 
suggested, TA systems are required to provide the learner with more learning choices and options. 
In addition, the system must allow the practitioners to flexibly design their own virtual learning 
environments which adapt to the learners' behaviors. For example, enhanced query input interfaces 
(e.g., conversation-based interface) and different types of learner control options would be more 
effective compared to the traditional TA systems (Yun, Choi, & Kim, 2008). Specifically, TA systems 
must aid the learner to flag the teaching process in which concepts and links are involved (Roscoe 
et al., 2008). 

Last, TA systems must be designed flexible enough to support learners' knowledge building 
processes. Essentially, the concept of TA systems includes a social aspect of mutual adaption. The 
learners observe the agent's answers and the learners revise the agent's knowledge and their own 
(Schwartz 8t Martin, 2006). In order to facilitate the mutual adaptation process, the system must be 
flexible. For example, an automatic pattern tracker can be utilized to determine whether an action 
or a series of actions match a predefined pattern (Leelawong et al., 2003). Particularly, advanced 
supportive ways for knowledge construction and collaborative learning must be invented and 
implemented for enhanced flexibility (Yun et al., 2008). 

In this project, a TA system, named EnTAM, was designed based on the design conditions reviewed 
above. The system uses a symbol manipulation method, which supports the learners in teaching the 
agent mathematics using number and math symbols. Once the learner manipulates the symbols 
through drag-and-drop, the pattern of the symbols is perceived by the system. 


Development 

The first stage of the development process involves building an online system architecture and its 
application framework using Web programming languages: JavaScript and PHP. The system consists 
of three parts: teaching, recognition, and test modules. First, in the teaching module, the learner 
drags and drops number icons and math symbols. The symbol manipulation interface enables the 
learner to communicate with the system. Specifically, as can be seen in Figure 1, draggable graphic 
icons were designed to serve as the learner-system communication tool. The learner uses these 
icons and manipulates them to teach the agent mathematics. 

Second, the recognition module perceives and recognizes the tutor's (i.e., actual learner's) teaching 
behaviors. Each graphic item is assigned to specific internal variables in the programming language 
so that the system can understand the meaning of each item. The system perceives learners' 
behaviors, and the learners' input is recognized by the agent system. Then, the system categorizes 
the teaching materials according to the predetermined patterns inside the system. 
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Last, in the result module, the patterns are related to the numbers of digit items. For example, "A" 
pattern includes the infinity symbol item (see Figure 9 and 11); "B" pattern three numbers of digit 
items; "C" pattern two number of digit items (see Figure 8); and "D" pattern one digit-item (see 
Figure 6). After categorizing the learners' input (i.e., teaching behavior), the test module shows the 
test result according to the analyzed patterns. 


Specification 

The TA system, named EnTAM, was developed for mathematics learning in the K-12 context. The 
learner teaches the TA using mathematics icons through drag-and-drop (see Figure 1, Right). The 
system acquires learners' teaching data through the representation of the icons that the learners 
manipulated. The learner's data is classified by the predefined patterns. Then, the agent makes its 
own decision based on the criteria and the teaching data when the test module is requested by the 
learner. 

Teaching phase. Figure 1 shows the teaching phase of the system. The learner teaches the TA using 
numbers (e.g, 1 to 9, "n", or "•••") and mathematical operators and/or symbols. 
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Figure 1. Teaching Phase (Left) and Draggable Items (Right). 

For example, the learners drag and drop the icons to explain the meaning of arithmetic mean. As 
can be seen in Figure 1, Right, the draggable items can be moved to the blackboard in the middle of 
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the screen. When an item is moved to an empty rectangle on the blackboard, the system perceives 
the meaning of the item. 

When a digit icon is moved to an empty rectangle area, the number of digits is generated 
automatically (see Figure 2, Left). For instance, if the learner used two number items (e.g., "1" and 
"2" in Figure 2), "the number of digits: 2" would be shown. The learner can also drag and drop this 
number on the blackboard to explain the topic (see Figure 2, Right). 


Chapter 1 


Mean = 

DHOBI 


The number of digits: 2 


Chapter l 


Mean = 


DQOQDB 


The number of digits: 


Figure 2. Automatic Generation of the Number of Digits (Left) and Dragging the Number (Right), 


If the "•••" item was located at the end of the parentheses, the infinity symbol (°°) would be 
generated (see Figure 3, Left). If the "•••" item was moved to the middle of the parentheses, the 
symbol of the digit number ("m") would be shown (see Figure 3, Right). 



Figure 3. Infinity icon (Left) and "m" icon (Right) as the Number of Digits 


When the "Let's call it a day" button (see Figure 1, Left) is pressed, the test phase is shown (see 
Figure 4). Then, a series of tests will be taken by the TA. 

Test phase. There are four example questions to measure the TA's knowledge, which is built on the 
human learner's teaching from the teaching phase. Figure 4 shows the screenshot of the test phase. 
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Figure 4. Test Phase 

When the "Question 1 " button on the left side of the screen is clicked, the test number 1 is answered 
by the TA. Then the answer is scored and graded automatically (see Figure 5). 
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’Chapter 1 
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Figure 5. Question 1 and its Result 
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Result type. Figure 6 shows the "Grade D" result type of the example instruction that was described 
in Figure 2. The "Grade D" result was generated when the learner used two numbers to explain the 
arithmetic mean to the agent. The TA who understands the mechanism of the arithmetic mean with 
two digits would only be able to answer the questions like "Question 1" at the test phase. Flowever, 
the agent will not be able to answer more complicated questions with the limited knowledge about 
the meaning of the mean. 



Figure 6. Result type - "Grade D" 

Figure 8 shows the "Grade C" result type of the example instruction described in Figure 7. The 
"Grade C" result type was generated based on the learner's teaching in which the three digits (i.e., 
3, 6, and 8) were used to explain the subject. The TA who understands the mechanism of the 
arithmetic mean with three digits can answer the Question 1 and 2 at the test phase. However, 
the agent's knowledge is limited to answer the complicated questions. 


Mean = 

DBCaQDDDB 


Figure 7. Example of "Grade C" Explanation 
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Teacher Eunyoung taught arithmetic mean as "(3+6+8)73" 

Eins took a test after your class. 

Eins' Grade : C 



Figure 8. Result type - "Grade C" 


Figure 10 shows the "Grade A" result type of the example instruction described in Figure 9. The 
"Grade A" result type was generated when the learner used the concept of infinity to explain the 
arithmetic mean. Since the TA is able to understand the mechanism of the arithmetic mean with 
the infinity, he can answer all the questions at the test phase. 

I(1i|+|4M M„T) 

Figure 9. Example of "Grade A" Explanation 
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Teacher Eunyoung taught arithmetic mean as "(1+4+...+n)/m" 
Eins took a test after your class. 

Eins' Grade : A 


Figure 10. Result Type - "Grade A" 

In addition, there is another "Grade A" result type. Figure 11, Right, shows another "Grade A" 
result type of the example instruction described in Figure 11, Left. 


Teacher Eunyoung taught arithmetic mean as "(4+...+2)/m" 
Eins took a test after your class. 

Eins' Grade : A 


Figure 11. Example of "Grade A" Explanation (Left) and the Result (Right) 


m± -In?! ) •••• 


Discussion 

This project describes the design and development process of a virtual learning-by-teaching 
environment. The system adopts a symbol manipulation method following design conditions 
suggested in the literature. The focus of the project was on addressing the challenges of the current 
Teachable Agent (TA) systems. The sample subject area of the proposed system is a part of basic 
statistics chapters found in middle school curriculum (i.e., arithmetic mean). 
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The main motivation for creating the TA system in this project was on the belief that from learning- 
by-teaching, people learn and assimilate their knowledge more effectively compared to direct 
instructions (Davis et al., 2003; Muis et al v 2016). In the proposed system, the learner teaches the 
TA arithmetic mean. Then, the virtual agent attempts to solve the questions with the knowledge 
that was built on the learner's teaching. The TA learns the meaning of the topic by generalizing the 
example demonstrated by the human tutor (i.e., actual learner) through manipulating numbers and 
mathematical symbols. Learner-computer communication methods are important in TA systems. 
This is because interaction is one of the most significant variables on impacting the students' 
learning through learning-by-teaching environments (Duran, 2016). The direct manipulation (e.g., 
drag-and-drop or multi-touch interaction) significantly supports young learners. Even pre¬ 
kindergarten children achieved high success rates of manipulation (Nacher, Jaen, Navarro, Catala, 
& Gonzalez, 2015). Therefore, effective interaction methods would help facilitate younger children 
to adopt TA systems (see Haake, Axelsson, Clausen-Bruun, & Gulz, 2015; Silvervarg, Kirkegaard, 
Nirme, Haake, & Gulz, 2014). Furthermore, natural language processing and conversation-based 
interaction approaches could also expand virtual learning-by-teaching environments (Azaria, 
Krishnamurthy, & Mitchell, 2016; Matsuda, Barbalios, Zhao, Ramamurthy, Stylianides, & Koedinger, 
2016). 

Another purpose of this project was to suggest a possibility of transforming the learning-by-teaching 
pedagogy into the virtual learning-by-teaching approach. Despite the success of current TA systems 
as revealed in different types of research, it is demanding to theorize the virtual learning-by- 
teaching pedagogy. The effectiveness of learning-by-teaching in computerized systems has not been 
generalized yet. This is partly due to challenges and the limited scalability of architectures in TA 
systems. In general, learning-by-teaching has been identified as an established pedagogical strategy, 
which is not limited to a specific learning domain. Therefore, the leaner-system interaction methods 
must become more diversified. 

This project proposed a virtual learning-by-teaching environment. Still, there are some limitations 
in this design and development project. First, the main limitation of this project is a lack of 
implementation efforts. Empirical studies using the proposed TA system are required for future 
research. Second, one of the ultimate goals of this project was to identify the extent to which the 
suggested agent system supports the learners in mathematics learning. Therefore, the learner's 
understanding must be examined in future research. Third, the proposed system was designed to 
support the learning of mathematics. When employing the virtual learning-by-teaching approach in 
mathematics, learners' prior knowledge must be considered (Bringula, Basa, Dela Cruz, & Rodrigo, 
2016). Future research needs to examine the effect of learners' prior knowledge in their learning 
through the proposed system. Fourth, TA systems are required to facilitate learning in a motivating 
and natural way for learners (Brophy, Biswas, Katzlberger, Bransford, & Schwartz, 1999). Thus, 
future research needs to investigate to what extent TA systems focus on learners' motivation by 
considering the individual difference in cognitive abilities and ongoing changes of the interest level. 
In sum, the following research questions are suggested for future research: (1) What evidence does 
the proposed TA system provide learner comprehension and problem solving in mathematics 
learning? and (2) How does the proposed TA system suggest a theory transforming from learning- 
by-teaching to virtual learning-by-teaching? 
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